
Claims 31, 39 and 45 have been canceled. 

Claims 51-56 have been added. 

Claims 1-30, 32-38, 40-44 and 46-56 are pending. 




Pursuant to 37 CFR §1.12 1(c)(1), please replace the current versions of 
claims 1-50 with the following clean copies of claims 1-30, 32-38, 40-44 and 

46-56: / 

/\\ One or more computer readable media having stored thereon a 
plurality ofSjistructions that, when executed by one or more processors, causes 
the one or mor& processors to perform acts including: 
receiving ah initial image selection; 

generating a plurality of query vectors by extracting, for each query 
vector, one of a plurality^f low-level features from the initial image selection; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature 
vectors corresponding to low-levePfeatures of a plurality of images; 

receiving feedback regarding relevance of one or more images of the 
set of potentially relevant images; 

generating a new plurality of query fetors based at least in part on the 
feedback; 

generating a weighting of feature element^based at least in part on the 
feedback; and 
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selecting a new set of potentially relevant images based at least in part 
on both the weighting of feature elements and distances between the new 
plurality of query vectors and the plurality of feature vectors. 
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A One or more computer readable media as recited in claim 1, 
wherein me selecting a new set of potentially relevant images comprises using 
a matrix in aetermining the distance between one of the new plurality of query 
vectors and one of the plurality of feature vectors, and further comprising 
dynamically selecting the matrix based on both a number of images in the set 
of potentially relevant images for which relevance feedback was input and a 
number of feature elements in the one feature vector. 

3. One or more computer readable media as recited in claim 2, 
wherein the dynamically Selecting comprises using a diagonal matrix if the 
number of images in the set of potentially relevant images for which relevance 
feedback was input is less tflan the number of feature elements in the one 
feature vector, and otherwise usmg a full matrix. 

4. One or more computer readable media as recited in claim 2, 
wherein the dynamically selecting comprises: 

if the number of images in thk set of potentially relevant images for 
which relevance feedback was input ia not less than the number of feature 
elements in the one feature vector, then using one matrix that transforms the 
query vector and the one feature vector to aVngher-level feature space and then 
using another matrix that assigns a weight tA each element of the transformed 
query vector and the transformed feature vectorc and 

if the number of images in the set of potentially relevant images is less 
than the number of feature elements in the one\ feature vector, then using a 
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matrix that\ssigns a weight to each element of the query vector and the one 
feature vectors 

5. (Amended) One or more computer readable media as recited in 
claim 2, wherein Jc\ represents an image matrix that is generated by stacking 
TV feature vectors, each of length K, corresponding to the set of potentially 
relevant images for which relevance feedback was received and resulting in an 
(N x K) matrix, C represents a weighted covariance matrix of X , det(C) 
represents the matrix determinant of C , and the matrix comprises a full matrix 
(W*) that is generated as fo^ows: 

fF* = (det(C))^C-\ 



6. (Amended) One or rr\ore computer readable media as recited in 
claim 2, wherein represents th^ kk ih element of matrix W 9 x k represents 
the k feature element, a k representsuhe standard deviation of the sequence of 
x k 's, the matrix comprises a diagonal matrix with each diagonal element ( w kk ) 
being generated as follows: 
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7. \ One or more computer readable media as recited in claim 1, 
wherein TV represents the number of images in the set of potentially relevant 
images for wftjch relevance feedback has been received, n n represents the 
relevance of im^ge n in the set of images, n represents a transposition of a 
vector generated tW concatenating the individual n n values, and X represents 
an image matrix \that is generated by stacking TV training vectors 
corresponding to the\set of potentially relevant images into a matrix, and 
wherein each new quei^( vector ( q ) of the new plurality of query vectors is 
generated as follows: 

- n X 

q = ^ — • 

n=l 



8. One or more computer readable media as recited in claim 1, 
wherein represents a summationAover the images in the set of potentially 
relevant images, of a product of a relevance of the image and a distance 
between the query vector and the feature vector, and wherein the selecting a 
new set of potentially relevant images comprises combining, for each image, a 
weighted distance between the plurality oAquery vectors and the plurality of 
feature vectors, and wherein the weight («M for each of a plurality (/) of 
distances between a query vector and a corresponding feature vector is 
calculated as: 

U i=2Ly\-f- 
j=i V Ji 
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9 A One or more computer readable media as recited in claim 1, 
wherein the receiving feedback comprises receiving feedback from a user. 

10. One or more computer readable media as recited in claim 1, 
wherein the low-level features include: a color moments feature, a wavelet 
based texture feature, and a water-fill edge feature. 

/l 1 . A method of selecting between two types of matrixes to be used 
to weight, based on refevance feedback, a plurality of feature elements for 
image retrieval, the methou comprising: 

selecting one of theVwo types of matrixes based on both a number of 
previously retrieved relevant Wages and a length of a feature vector including 
the plurality of feature elements 

12. A method as recited hi claim 11, wherein the selecting comprises 
selecting one of the two types of matrixes based on both a number of 
previously retrieved potentially rele^nt images which were identified by a 
user as being relevant, and the length of the feature vector including the 
plurality of feature elements. \ 

13. A method as recited in claim \ l, wherein the plurality of feature 
elements are all elements of the same featureA 
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14\ A method as recited in claim 11, wherein the selecting comprises 
using a firstW>e of matrix if the number of retrieved relevant images is less 
than the lengtn^of the feature vector, and otherwise using a second type of 
matrix. 

15. A methcJd as recited in claim 14, wherein the first type of matrix 
comprises a diagonal matrix and wherein the second type of matrix comprises 
a full matrix. 

16. A method as recited in claim 11, wherein the selecting comprises 
using a first type of matrix if the length of the feature vector exceeds the 
number of retrieved relevant images by at least a threshold amount, and 
otherwise using a second type of mamx. 

17. A method as recited in claWi 16, wherein the first type of matrix 
comprises a full matrix and the second t^pe of matrix comprises a diagonal 
matrix. 



18. One or more computer readable\ media including a computer 
program that is executable by a processor to perform the method recited in 
claim 1 1 . 
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A 19. \( Amended) One or more computer readable media having stored 
thereon a plurality of instructions that, when executed by one or more 
processors, causses the one or more processors to perform acts including: 

comparing^ for each of a plurality of images, a plurality of feature 
elements from a query vector to a plurality of feature elements from a feature 
vector correspondingyto the image; 

identifying a number of potentially relevant images based on the 
comparing; \ 

receiving user feedback regarding relevancy of one or more of the 
potentially relevant images; \ 

re-comparing, for eachW the plurality of images, the plurality of feature 
elements from the query vector to the plurality of feature elements from the 
feature vector, including using a matrix to compare the feature elements and 
dynamically selecting a type of matrix to use based on both the user feedback 
and the number of the plurality of feature elements; 

identifying a new set of potentially relevant images based on the re- 
comparing; and \ 

presenting the new set of potentially relevant images to the user. 

20. One or more computer readable media as recited in claim 19, 
wherein the re-comparing comprises dynamically selecting the type of matrix 
to use based on both a number of the potentially relevant images for which user 
feedback has been received and the number of the plurality of feature elements. 
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21. \One or more computer readable media as recited in claim 19, 
wherein the dynamically weighting comprises using a first type of matrix if the 
number of retrieved relevant images is less than the length of the feature 
vector, and otherwise using a second type of matrix. 

22. One onmore computer readable media as recited in 21, wherein 
the first type of matnx comprises a diagonal matrix and the second type of 
matrix comprises a full matrix. 



/23 . A method comprising: 
generating a query veitor corresponding to a feature of one image; 
identifying a feature vector corresponding to the feature of another 
image; \ 

identifying a number of training samples for which relevance feedback 
has been received; \ 

if the number of training samples either equals or exceeds a threshold 
amount, then determining a distance\between the query vector and the feature 
vector including transforming the query vector and the feature vector to a 
higher-level feature space and then assigning a weight to each element of the 
transformed query vector and the transformed feature vector; and 

if the number of training samples d&es not exceed the threshold amount, 
then determining the distance between the query vector and the feature vector 
including assigning a weight to each element of the query vector and the 
feature vector. \ 
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24. A method as recited in claim 23 , wherein the feature vector 
includes A plurality of feature elements and wherein the threshold amount 
comprises tWe number of feature elements in the feature vector. 

25. A\nethod as recited in claim 23, wherein if the number of 

training samples Neither equals or exceeds the threshold amount, then 

\ ~* 
determining the distance (g), where P is a mapping matrix, q is the query 

vector, jc is the featureyector, and A is a weighting matrix, as: 
g = (P(q-h T M^q-h^ 

26. A method as recited in claim 23, wherein if the number of 
training samples does not exceed the threshold amount, then determining the 
distance ( g ), where q is the querv vector, x is the feature vector, and A is a 
weighting matrix, as: \ 

g = (q-x) T A(q-x). \ 

27. (Amended) A method as recited in claim 23, further comprising: 
repeating the generating, identifyingW the feature vector, identifying of 

the number of training samples, and the determining for each of a plurality of 
features; and \ 

identifying how closely the image and me another image match each 
other by combining the distances between the query vectors and the feature 
vectors for the plurality of features. \ 
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2& (Amended) A method as recited in claim 27, wherein the 
identifying how closely the image and the another image match each other 
comprises calculating a weighted summation of each of the individual 
distances for eJich of the plurality of features. 

29. One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in 
claim 23. \ 



a query vector generator to generate a query vector corresponding to a 
feature of one image; \ 

a comparator, coupled toVhe query vector generator, to, 
identify a feature vector corresponding to the feature of another image, 
identify a number of training samples for which relevance feedback has 
been received, \ 

if the number of training samptfes either equals or exceeds a threshold 
amount, then to determine a distance between the query vector and the feature 
vector including transforming the query Vector and the feature vector to a 
higher-level feature space and then assigning a weight to each element of the 
transformed query vector and the transformemfeature vector, and 

if the number of training samples does ixpt exceed the threshold amount, 
then to determine the distance between the queiy vector and the feature vector 
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including assignmg a weight to each element of the query vector and the 

* 32\(Amended) A method comprising: 
for one of a plurality of images and each of a plurality of features, 
generating, based on the set of search criteria, a query vector for the 
feature, \ 

identifying k feature vector, corresponding to the image, for the feature, 
and \ 

determining how\closely the feature vector matches the query vector; 
and \ 

determining how closedy the image matches the set of search criteria 
based on how closely, for the plurality of features, the feature vectors match 
the query vectors, wherein generating the query vector comprises generating 
the query vector based at least in part on user relevance feedback regarding 
how relevant images previously displayed to a user were. 

33. (Amended) A method as reciteei in claim 32, wherein identifying 
the feature vector comprises: \ 

identifying a low-level feature vector corresponding to the feature; and 
mapping the low-level feature vector to a higher level feature space. 
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34. A method as recited in claim 33, wherein the identifying the 
feature vector further comprises incorporating, into the mapping, relevance 
feedback. \ 

35. (Amended) A method as recited in claim 32, wherein the initial 
search criteria comprises an image. 

36. (Amendech A method as recited in claim 32, wherein the 
determining how closely the feature vector matches the query vector comprises 
determining a distance between the feature vector and the query vector, and 
wherein the determining how closely the image matches the set of search 
criteria comprises calculating k weighted summation of each of the individual 
distances between the feature vectors and the query vectors. 

37. A method as recited Vn claim 36, wherein the calculating a 
weighted summation comprises calculating the weighted summation based at 
least in part on user relevance feedback regarding how relevant images 
previously displayed to a user were. \ 

38. (Amended) One or more computer readable media including a 
computer program that is executable by a processor to perform the method 
recited in claim 32. \ 

P lca c o oan o ol claim 39 wi l lio ul' pi cjii d're e* 
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40. (Amended) One or more computer readable media as recited in 
claim 41 X wherein the identifying the plurality of query vectors comprises 
extracting ttte plurality of query vectors from the image. 



/41. (Amended) One or more computer readable media having stored 
thereon a pluralitV of instructions that, when executed by one or more 
processors, causes the\pne or more processors to perform acts including: 

identifying a plurality of query vectors for one image, each query vector 
corresponding to one of a plurality of features; 

identifying a pluralityvof feature vectors for another image, each feature 
vector corresponding to one oMie plurality of features; 

for each feature, determining a distance between the corresponding 
query vector and the correspondingyfeature vector; and 

combining the distances to generate a value representing an overall 
distance between the one and the another image, wherein the identifying the 
plurality of query vectors comprises generating the plurality of query vectors 
based at least in part on user relevancev feedback regarding how relevant 
images previously displayed to a user were. \ 

42. (Amended) One or more compute^ readable media as recited in 
claim 41, wherein the determining the distance\ between the corresponding 
query vector and the corresponding feature vector mcludes incorporating, into 
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the detennining, user relevance feedback regarding how relevant images 
previously displayed to a user were. 

43. (Amended) One or more computer readable media as recited in 
claim 41 , wherein the combining the distances comprises calculating a 
weighted summation of\ach of the individual distances between the feature 
vectors and the query vectors^ 

44. One or more compWr readable media as recited in claim 43 , 
wherein the calculating a weighted summation comprises calculating the 
weighted summation based at least\in part on user relevance feedback 
regarding how relevant images previously displayed to a user were. 

fil caG C- can o ol claim 15 without projudio© *. 

/ 46v A method of generating a query vector to compare to a feature 
vector of anothfe^image, the method comprising: 

receiving feeab^ck regarding the relevance of each image of a set of 
images; 

wherein TV represents the'number of images in the set of images for 
which user relevance feedback has beeiNeceived, n n represents the relevance 
of image n in the set of images, n represbqts a transposition of a vector 
generated by concatenating the individual n n values, and X represents an 
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image matrix that is generated by stacking N training vectors corresponding to 
the set of images into a matrix; and 

generating a query vector (q) corresponding to one of a plurality of 
features as follows: 



- n X 



n=\ 



47. One or more opmputer readable media including a computer 
program that is executable hy\ processor to perform the method recited in 
claim 46. 



48. A method of generating V weight to apply to distances between 
query vectors and feature vectors when \jombining the distances, the method 
comprising: 

receiving feedback regarding the reliance of each image of a set of 
images; 

wherein f i represents a summation, oVer the images in the set of 
images, of a product of a relevance of the image^ and a distance between the 
query vector and the feature vector; and 
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generating a weight ( u i ) for each of a plurality ( / ) of distances between 
a query vector corresponding to one of a plurality ( / ) of features and a feature 
vector corresponding to the one of the plurality (/ ) of features as: 

j=\ V Ji\ 

49. One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in 
claim 48. \ 

/50. A system compnking: 
a client device; \ 
a collection of a plurality of images; 

an image server, coupled ta the client device and the collection of a 
plurality of images, the image server to receive image retrieval requests from 
the client device and to, \ 

receive an initial image selection Yrom the client device, 

generate a plurality of query vectors by extracting, for each query 
vector, one of a plurality of low-level features from the initial image selection, 

select a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature 
vectors corresponding to low-level features of a plurality of images, 

receive feedback regarding the relevance ©f one or more images of the 
set of potentially relevant images, \ 
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generate\a new plurality of query vectors based at least in part on the 
feedback, 

generate a weighting of feature elements based at least in part on the 
feedback, and 

select a new set 6f potentially relevant images based at least in part on 
both the weighting of feature elements and distances between the new plurality 
of query vectors and the plurality of feature vectors. 

New claims: 

(New) One or more computer readable media as recited in claim 1, 
wherein the\receiving feedback comprises receiving feedback in a range 
including at lea^t Highly Relevant, Relevant, No Opinion, Irrelevant, and 
Highly Irrelevant. 

52. (New) One or more computer readable media as recited in claim 1, 
wherein the receiving feedbabjc comprises receiving feedback via speech 
recognition. 



53. (New) One or more compute\readable media as recited in claim 
19, wherein the receiving user feedback Vegarding relevancy comprises 
receiving user feedback in a range including at least Highly Relevant, 
Relevant, No Opinion, Irrelevant, and Highly Irrelevant. 
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ii4. (New) One or more computer readable media as recited in claim 
19, whereV the receiving user feedback comprises receiving user feedback via 
speech recognition. 



/55. (NewJvOne or more computer readable media including a computer 
program that is executable by a processor to cause the processor to perform 
acts of: 

receiving feedback regarding the relevance of each image of a set of 
images, the user feedbaclAforming a range including at least Highly Relevant, 
Relevant, No Opinion, Irrelevant, and Highly Irrelevant; 

wherein AT represents the number of images in the set of images for 
which user relevance feedback has been received, n n represents the relevance 
of image n in the set of imagfes, n represents a transposition of a vector 
generated by concatenating the ^individual rc n values, and X represents an 
image matrix that is generated by slacking N training vectors corresponding to 
the set of images into a matrix; and 

generating a query vector (#)\corresponding to one of a plurality of 
features as follows: 



- K X 
<1= — 
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56. (NewO One or more computer readable media as recited in claim 
55, wherein the receiving user feedback comprises receiving user feedback via 
speech recognitionA 
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